The present invention is in the field of three-dimensional image capture and in particular to the capture of three-dimensional image information with a scannerless range imaging system.
Standard image capture systems will capture images, such as photographic images, that are two-dimensional representations of the three-dimensional world. In such systems, projective geometry best models the process of transforming the three-dimensional real world into the two-dimensional images. In particular, much of the information that is lost in the transformation is in the distance between the camera and image points in the real world. Methods and processes have been proposed to retrieve or record this information. Some methods, such as one based on a scanner from Cyberware, Inc., use a laser to scan across a scene. Variations in the reflected light are used to estimate the range to the object. However, these methods require the subject to be close (e.g., within 2 meters) to the camera and are typically slow. Stereo imaging is a common example of another process, which is fast on capture but requires solving the xe2x80x9ccorrespondence problemxe2x80x9d, that is, the problem of finding corresponding points in the two images. This can be difficult and limit the number of pixels having range data, due to a limited number of feature points that are suitable for the correspondence processing.
Another method described in U.S. Pat. No. 4,935,616 (and further described in the Sandia Lab News, vol. 46, No. 19, Sep. 16, 1994) provides a scannerless range imaging system using either an amplitude-modulated high-power laser diode or an array of amplitude-modulated light emitting diodes (LEDs) to completely illuminate a target scene. Conventional optics confine the target beam and image the target onto a receiver, which includes an integrating detector array sensor having hundreds of elements in each dimension. The range to a target is determined by measuring the phase shift of the reflected light from the target relative to the amplitude-modulated carrier phase of the transmitted light. To make this measurement, the gain of an image intensifier (in particular, a micro-channel plate) within the receiver is modulated at the same frequency as the transmitter, so the amount of light reaching the sensor (a charge-coupled device) is a function of the range-dependent phase difference. A second image is then taken without receiver or transmitter modulation and is used to eliminate non-range-carrying intensity information. Both captured images are registered spatially, and a digital processor is used to operate on these two frames to extract range. Consequently, the range associated with each pixel is essentially measured simultaneously across the whole scene.
The preferred method of estimating the range in the ""616 patent uses a pair of captured images to be captured, one image with a destructive interference caused by modulating the image intensifier, and the other with the image intensifier set at a constant voltage. However, a more stable estimation method uses a series of at least three images, each with modulation applied to the image intensifier as described in commonly assigned copending application Ser. No. 09/342,370, entitled xe2x80x9cMethod and Apparatus for Scannerless Range Image Capture Using Photographic Filmxe2x80x9d and filed Jun. 29, 1999 in the names of Lawrence Allen Ray and Timothy P. Mathers. In that application, the distinguishing feature of each image is that the phase of the image intensifier modulation is unique relative to modulation of the illuminator. If a series of n images are to be collected, then the preferred arrangement is for successive images to have a phase shift of       2    ⁢    π    n
radians (where n is the number of images) from the phase of the previous image. However, this specific shift is not required, albeit the phase shifts need to be unique. The resultant set of images is referred to as an image bundle. The range at a pixel location is estimated by selecting the intensity of the pixel at that location in each image of the bundle and performing a best fit of a sine wave of one period through the points. The phase of the resulting best-fitted sine wave is then used to estimate the range to the object based upon the wave-length of the illumination frequency.
A drawback of methods using an image intensifier is that color information is lost. Unfortunately for color applications, an image intensifier operates by converting photonic energy into a stream of electrons, amplifying the energy of the electrons and then converting the electrons back into photonic energy via a phosphor plate. One consequence of this process is that color information is lost. Since color is a useful property of images for many applications, a means of acquiring the color information that is registered along with the range information is extremely desirable.
One approach to acquiring color is to place a dichromatic mirror in the optical path before the micro-channel-plate. Following the mirror a separate image capture plane (i.e., a separate image sensor) is provided for the range portion of the camera and another image capture plane (another sensor) is provided for the color texture capture portion of the camera. This is the approach taken by 3DV Technology with their Z-Cam product. Besides the added expense of two image capture devices, there are additional drawbacks in the need to register the two image planes precisely, together with alignment of the optical paths. Another difficulty is collating image pairs gathered by different sources.
Recognizing that the system described in the ""616 patent may be implemented in relation to a normal camera system, and, in particular, that a standard camera system may be converted into a range capture system by modifying its optical system, another approach is to capture an image bundle having interchangeable optical assemblies: one optical assembly for the phase image portion and a separate optical element for the color texture image portion. This approach is described in detail in commonly assigned copending application Ser. No. 09/451,823, entitled xe2x80x9cMethod and Apparatus for a Color Scannerless Range Image Systemxe2x80x9d and filed Nov. 30, 1999 in the names of Lawrence Allen Ray, Louis R. Gabello and Kenneth J. Repich. The drawback of this approach is the need to switch lenses and the possible misregistration that might occur due to the physical exchange of lens elements. There is an additional drawback in the time required to swap the two optical assemblies, and the effect that may have on the spatial coincidence of the images.
A scannerless range imaging camera may operate either as a digital camera or a camera utilizing film. In the case of a film based system there are some other requirements that need to be met. These requirements and means for satisfying them are described in the aforementioned copending application Ser. No. 09/342,370. As mentioned above, the drawback of such a camera system, including a film-based system, is its inability to capture a color image.
What is needed is a convenient camera system that would avoid the aforementioned limitations and capture ranging information without sacrificing color information that would otherwise be available for capture.
It is an object of the invention to provide a scannerless range imaging system that is capable of capturing both range images and color images of a scene.
The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, a color scannerless range imaging system includes an illumination for controllably illuminating a scene with modulated illumination and an image responsive element for capturing image light from the scene, including the modulated image light. The system establishes a primary optical path for directing image light toward the image responsive element. A beamsplitter located in the primary optical path separates the image light into two channels, a first channel including an infrared component and a second channel including a color texture component, whereby one of the channels traverses a secondary optical path distinct from the primary path. A modulating element is operative in the first channel to receive the infrared component and a modulating signal, and to generate a processed infrared component with phase data indicative of range information. An optical network is provided in the secondary optical path for recombining the secondary optical path into the primary optical path such that the processed infrared component and the color texture component are directed to the image responsive element.
The current invention eliminates the requirement for two image capture planes, but allows the operator to collect a full range map with texture with a single exposure activation. In particular, the invention employs a dichromatic mirror to separate a channel for capturing range data and a channel for capturing color texture data, while only using a single image capture subsystem. The advantage of the present invention is that is provides a means of obtaining a color image along with range information for each point on the image. Besides using a scannerless range image capture method, which is rapid and operative over a longer range than other methods, the invention includes a single optical assembly and a dual illuminating system so that both range and color information may be efficiently captured under the best illuminating conditions for each capture. The ability to accomplish this is provided in part by having the range capture system embodied as a camera attachment that is optically coupled with the image capture device.
These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiments and appended claims, and by reference to the accompanying drawings.